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OBJECTIVES 

The objective of this study was to investigate the effectiveness of a 
feedback technique which focuses on a specific skill (FSS) relative to 
a feedback method using an ordered category observation system. The FSS 
was obtained using the microteaching method. In addition the effectiveness 
of the two methods when combined was also tested. 

The different methods were applied in the training of student teachers 
in order to achieve a change in teaching behavior according to the following 
two behavioral objectives: 1) To redirect the teaching emphasis from a teacher- 
centered one to a student-centered one by increasing student participation; 
and 2) to improve the level of instruction by stressing analytical and creative 
thinking rather than Just the absorption of knowledge. 

THEORETICAL FRAMEWORK 

The combined use of microteaching and category observation systems 
is highly recommended by its developers and users (Allen, 1969; Flanders, 1971). 
Many studies have been conducted to investigate and evaluate the relative 
effectiveness of the two systems when usea separately and also in examining 
the interaction between them when used together (Rosenshine and Furst, 
Second Handbook of Research on Teaching . ) . 

The difference between the methods is that in microteaching, the training 
in each lesson is focused on a specific skill and the training objective is to 
perfect the performance of that skill. The feedback analysis is based on a 
sign system in which the observer counts the number of times the specific 
behavior occurs and reports the total to the trainee. 

In a category observation system emphasis is placed on changing the 
structure of the lesson as a whole and the ratio of various behaviors of 
the teacher and the student are analysed. Ail behaviors are recorded, 
organized in a matrix and ratios are established between the frequencies 
of the different categories. For this technique the purpose of the training 
is to change these ratios. Perlberg et al (1971, 1972) have previously 
described an ordered category observation known as the Technion Diagnostic 
System (IDS) which is based on Guttman's Facet Theory (Bar-On and Perlberg, 
1973). A brief description of the TDS follows. The TDS includes 24 
l^y^gorles that are all the possible permutations of two seta. One set 
fflMaaina 8 categories which describe the teaching activities (FACET A) 
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and the second set contains three categories that describe the level of 
thinking (FACET B) . The categories are: 



FACET A 

a^ - teacher lectures verbally 

- teacher lectures non-verbally 

- teacher gives instructions 
a^ - teacher asks questions 
a^ - teacher responds to pupil's reactions 
ag - pupil responds 

aj - teacher relates to pupil's initiative 
ag - pupil initiates 

The TDS was planned so that^ in FACET A, teaching activities are arranged 
according to the amount of pupil participation, from a. (low) to (high). 
In FACET B levels of thinking are arranged according to increasing levels 
of pupil intellectual stimulation from b^(low) to b2(high). It should be 
noted that both facets are arranged from low to high pupil stimulation. 
The resultant Cartesian set which is partially ordered, enables computation 
of a general score for each lesson. Students ware instructed to plan their 
lessons in order to achieve the highest possible score. 



FACET B 

bj^ - knowledge 

b2 - analytical thinking 

b^ - creative thinking 



METHODS AND TEC H NIQUES 



The two methods of feedback mention^ed were manipulated, and the following 
factorial design was used: 





YES 


NO 


Focusing on 

a Specific Skill (FSS) 


n S 32 
(3) 


n -,32 

(2) 




NO 


32 
(1) 


n Z.22 
(A) . 





* ^n= number of subjects; numbers in parenthesis designate 
treatments as defined below. 



A more detailed description of the TDS and the way it was structured can be 
.gj^Qid In "The Facet Approach in Developing a Theory of Instruction", Bar-On, 
"fSfiberg, 1973. 
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The four treatments were the tollowLng: 
Focus on a specific skill (treatment 1) 

The supervisor focused on the specific skill practiced during the lesson, and 

a discussion on how the skill was performed by each trainee took place. 

The skills practiced were set induction, probing (juestions, high order questions, 

the use of examples, and the use of audio-visual aids. 

TPS feedback (treatment 2) 

Supervision was focused on the relative frequencies of the various teaching 
behaviors and the calculated ratios between Lhem. For this purpose the 
supervisor was supplied with a computer printout containing the various 
frequencies and ratios. Tlie scores calculated expressed the amount of 
pupil participation and the level of thinking. The discussion aimed at how 
to increase these scores- 

Focusing on a specific skill m combination v;ith TPS (treatment 3; 
Emphasis was placed on specific skills, but the trainee was also told to 
concentrate on pupil participation and the level of thinking. During the lesson^ 
each three-second unit which was coded on a time- line display was classified 
by two peer observers according to one of the 24 categories of the IDS. 
Supervision focused on type of interaction between the trainee and Lhe 
pupils and the use of high thinking levels in teaching the skill which 
was conducted that dayr Here the skills that were mentioned under treatment 2 
were practiced. 

General supervision (treatment 4) 

Supervision was neither focused on a specific skill nor on relative frequencies 
and ratios. The discussion consisted of general comments and recommendations 
for improvement. Both trainees and supervisor had in mind the general 
idea of changing the trainee behavior In the directions specified earlier 
but they did not act systematically towards this end. 



THE TEST POPULATION 



The test population consisted of students from the Teacher Training 
Department at the Technion, all of whom were taking the course Principles 
of Teaching , and were being trained to be science teachers » The same course 
is repeated every year, and in each of the four semesters starting with semester 
II. ^971, and ending semester I, 1973, all tlie mentioned four treatments were 
^ERIC "^he students in the four treatment groups differed only slightly (some 
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of them were on their first semester in the course and otiiers in the second 
semester) and the investigators assumed that this small difference in the 
background of the trainees had no great influence on the results of the investigation. 
The whole sample consisted of J28 students - 32 in eacli semester. Tha 32 
students that made up each treatment group were chosen randomly from about 80 
students participating in the course, 

THE TRAINING SITUATION 

In order to limit the variation among treatments, the training method 
consisted of using a microteaching laboratory with a group of 4 students 
and a supervisor in each room. Every student taught one microlesson per week, 
in the subject in which he was majoring. 

The microlesson lasted five minutes and was taught to five 7th grade 
pupils. For the 15 minutes following the trainee's own lesson he received 
a critique from the supervisor whose comments related to the objectives of the 
specific treatment group. In addition the student received feedback via 
comments by other students in his group and by seeing himself on television. 
Furthermore each trainee participated in the supervisory discussion of the other 
three trainees in his group. 

Although not all the supervisors were the same ones during the whole 
period the investigations lasted, the core of the supervisors did remain the 
same. In order to limit the possibility that any type of bias on the part 
of the supervisors might affect the results, all the super\xsors were trained 
in the specific method on which they concentrated and in addition they were 
provided with written instructions specifying how sessions should be conducted. 
From time to time they were videotaped in order to check whether their methods 
of supervision coincided with the intent of the investigations. In addition 
to all these means of eliminating the personal effect, a rotation system 
was also applied, so that each semester all supervisors worked with each group of 
4 students. 
Manipulated Variables 

The use of TDS and tlie use of feedback focusing on a specific skill 
as a feedback mechanism - were the two factors manipulatedv 



er|c 
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Cons t - mr. Var i. » b I c s 

All the students m ilic i''ur Lrentmont Kt'oup.s had ..i simi lai (even though not 
identical) back^^ -'^•^'i^^ in oilur .w. L.'n and psycho loj;v , i.hc mctiiods o\' training were 
iiiadG uniform so thai tln)v^ w:n^ hi ho n,- cilccL of supervise; bus and the 
.same lecturer laugh- the r/»r:sc, Pr mcip los of ^^'^-'^ ^-h i n^ , during ihe entire time 
the research was taking pla:e 

Data Sourc e 

In all cases ihe first (pretest) and last (pasctest) vide,- raped 
lessons of each student were cc^ded according to li^o Techniun Diag^nostic 
System (T.D.S.)^ The [ ive~mi,nute lessons were divided into 100 units of 
3 seconds each, and every unit was classified inco one cf the 24 possible 
categories. The coding was done by three trained observers The same instructions 
were given to the scucJen^s m all four treatment groups both for the pretest and 
the posttest. 

It was explained to the students tha': they should give a live-minute 
lesson and cry to make it the best thai thoy coul<i give. A General Score - 
(GS)was calculated according to the following formula:"" 

GS= K-*^ K^PT E X.Y. where 
12 11 

- percentage of lesson time of catcQ;ory i (0;^ X^.<_ 100) 

Y. - score of category i ( i Y JQ^ "5^ 
1 J. 

P - proportion of pup i is participating m the lesson 

T - a variable chat teflectt.- the number ol transactions f : :m v:;ne category to 
another , 

- 40 
K^pL- 0, 125 

In addition to GS . t he frequency of each of the 24 categories v/as calculated 
for each group and it was -found liiat 8 combinations occuz.ed i;. .s. ircquently in 
all four groups- Tficse 8 ratcgories aioae covered at least 80% of each lesson^ 



1) A more detailed description of tiie structuring of this scjre, Its 

distribution and its significance can be found iii "The Use oC Computers in 

Evaluating Teacher Competency" Submitted to A.EcR.A., April, 1974> 

*2) The structuple a b was rated the lowest and given the gradt. of 1^ ^Qb was 

11 o J 

rated the nighest and given the grade of 10 and the other 22 siructuples were 
Q rated in relation to their position to the two structuples mentioned above. 
E^^CLncomparable structuples (e.gr a^b^^ and '^j^^^) get tlie same grade. 



aria since they also reflect the amount of participation of pupils in the 
lesson and the Le^^el of ^.hLnking^ ic was decided to use only these for the 
evaluation and ignore the ether 16 caLcgoLics 
The 8 categories referred to .ibove were: 

Category score 



1 - 


ieccuie jn knowledge it;vcl ^-^^^^ 


1 


2 - 


lecture on analytical lovc-i ) 


2 


3 - 


knowledge questions ^^b^ 


4 


4 - 


analytical question J^^^ -, 


5 


5 -• 


teacher rea::t,ion - kncwledge ^^^^ 


5 


6 - 


teacher reaction - anaiyi ic '""'3^2 


6 


7 - 


pupils response = kn .wlcdRe n,b, 

6 1 




8 - 


P'apil response - analycic =^^1^2 


7 



Each student was given 9 scores — 8 cl" Lhcni were the frequencies of 
above mentioned eight cstegcries plus one general score - OS. The 
same scoring technique was applied boih for pretest and posttest. 

Results 

There was an a priori assumption that a linear relationship between the 
pretest and posttest exis-.^d and therefore <in analysis of covariance (BMDX 69 
computer program) was tried. In all cases the score of the pretest was used as 
the covarlate and the posctes^: S':cre as the dependent variable. It was 
hoped Lhat if such llneacity existed becter precision might be attained by 
the posttest score accocding to the regression coefricient 

(].ns-.rt table 1) 

As can be seen from table 1, che F sta'^iSLic tor the MS covarlate - 
MS error rntio was Jow and insignif icanc m all cases. The regression coefficients 
under eacii hypothesis show us chao chcrc are no linear relationships between 
pretest and posttest scores, a conclusion alrccidy reached in a previous study 
(Perlberg, et al. 1973) 

Since no linear relationship between the pretest and the posttest 
was found, regular analysis of variance (ANOVA) was performed on the pretest 
scores in order to find out whether there were any initial diff^Brences in the 
scores of the four treatment groups. The AIJOVA revealed no significant 
differences - and since there were no differences in the background of the 
Rir^ groups, this result is quite understandable. 
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Because there were no d j I f uncus between the gi*oups» the posttcsts 
were also analysed by cinalysis oi variances Table 2 sumniar i;!cs the effects 
of the Focusing on a Specific Skill treatment (FSS),the TDS treatment, and the 
interaction becween them< IL also gives the statistic l/- which is the percent 
of the variance that is explained by each factor. Since the V test is 
non-direcLional, the arithmetic means are also listed in order to enable 
the reader to see the direction of the difference between the four treatments. 

(insert table 2) 

The results can be summarized as follows: 
1. The most efficient technique fur the decreasing the time devoted tu 
lecturing on the kiiowlcd,^;e level is the use of the TDS The training by 
the TDS decreased this score ro an average 13 3% of the lesson, while the combined 
use of tlie specific skill treacment with TDS decreased it only to 25,7%. 
The use of FSS alone resulted in an average of A] . 2% of lecturing on tlie 
knowledge so that the difference ljetv;ecn the effectiveness of the two treatments 
for decreasing this score is quite abvious. 

The effect of the TDS is significant ( p^ 0,01) and it explains 26% 
of the total variance- 

2- Training by the TDS resuUo ."in the greatest amount of iecturJ.ng on 
the analytical level (x = 12.2%) and the TPS combined with FSS elves better 
results than FSS alone. (x - ^'^ D'Z ami 1 7/^ respectively) Jjuth the effects 
of TDS and interaction were sigu.i Licarit , and the TDS explains 19% of the 
variance, the interaction 6%. and FSS f' niy 3X 

3' FSS has a significant effect oa increasing knowledge questions, and the 
combined use with TDS gives ilie be:>L results although TDS alone has no 
effect. The FSS explains 10% of the variance and the interaction - 8% only. 

4. In increasing the frequency or analytical questions, TPS is the only 
efficient technique, while FSS has ncj significant effect when applied 
alone or with the TDS. Tlie TDS expjains 23% of the variance, and the 

mean performance of analytical questioning is 12.3% under the TDS treatment, 
and only 3.2% under FSS, 

5. In teacher reaction to pupils, resp:vasc on the knowledge level, both the TDS 
and FSS have significant effr:c:ts, but i/niie the FSS treatment increases this 
score, the treatment, by TDS decreases it. The means are 4,5% for TDS 

. 9% for FSS. There is no significant interaction effect. 
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6. For teacher reaction on the analytical level, the TDS has a very significant 
effect (it explains 38% of the variance) of increasing the frequency, while FSS 
has no effect at all. The mean for the TDS treatment is 6.9% while the mean 
for FSS is 0. 7% cnly. 

7. Pupils' response in the Icvei of knowledge is affected mainly by the 
TDS (explains 20% of the variance) but it seems that this effect is in the 
negative direction, that is, it decreases the frequency of this category. 
Although FSS has no significant effect, the mean for the FSS treatment is the 
highest (x = 22.3%) while the mean for the TDS treatment is 10.5% only. 

8. In increasing the frequency of pupils* response on the analytical level, 
the TDS is the only factor that has any effect at all, it explains 32% of 
the variance and the mean for this treatment group is 18.6% The mean for 
the FSS treatment is only 2.6% and it explains only 3% of the variance. 

The interaction effect is also significant and explains 7% for the variance. 

9. Ulien a general score of i)upil stimulation is calculated, the main effect 

is caused by the TDS and ^even though the interaction effect is also significant, 
the mean for the treatment ' by TDS (x=^ 90) is much higher than that for the 

combined treatment (x=76)« The mean for the FSS treatment group was only 72. 

To sum up the results it can be said that all scores measuring activities 
on the analytical level of thinking, were higher under the TDS treatment, while 
the activities on the level of knowledge (except for lecturing) were more 
frequent under the FSS treatment. 

The only score which was higher under the combined treatment was the 
score for knowledge questions. 

DISCUSSION 

Since the study was carried out over a period of three years, it was 
impossible to keep all the environment variables constant, and the different 
treatments were given to students of somewhat different backgrounds by 
different supervisors- Furthermore, because of the various possible treatments 
that could be applied, the investigators decided to try a new one each 
semester instead of repeating a treatment that had been tried already - thus 
there is no replication for this study. 

Nevertheless, the effects of the treatments were significant and for 
O f the calculated scores the percentage of variance explained by the 




ilated variables was quite large. Therefore it seems that the conclusions 
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of the study are indeopndenl: of a LI. the hackground noise. 

The results !uive shown that for technical skills such as number of questions 
or teacher response on the knowlQd|.;e level - the training that breaks up the 
instructional process into specific skills and focuses cn each skill separately 
had the effect of changing the beiiavior - and increased tlie pri^i)ortion of that 
skill in the whole lesson. Nevertheless, for objectives of higher order, like 
increasing pupil's participation and at the same time his level of thinking, 
tlie TDS as a feedback mechanism had better results, even better than the 
combined use of both techniques. 

A theoretical analysis of a pactern of a good lesson would reveal the 
following: In order to achieve a lesson in which both the amount of participation 
and the level of thinking are high - the teacher has to plan it so that all the 
activities - from Jow co high on the pupil participation scale, would involve 
higher level thinking. Since the length of a lesson is fixed, it is impossible 
to increase all the scores, and increasing part of the scores actually means 
decreasing the rest of them^ So that analytical responses to his questions 
would be achieved, the teacher has to increase the frequency of his analytical 
questions - thus decreasing the amount of knowledge questions. In order to 
be able to ask analytical questions - he also has to increase the amount of 
analytical lecturing - even if lecturing itself gets only a low grade. The 
frequency of the teacher's reactions to pupils on the level of knowledge also 
has to decrease so that he can increase the amount of liis analytical reactions. 

Focusing on a specific skill in each session does increase the practice 
of that skill in the posttest lesson, but the lesson as a whole remains on a 
lower level cf thinking, since inducing analytical thought cannot be 
practiced technically. Only if the final objective of the training iij 
emphasized during each stage of the treatment, are the desired results achieved. 
The results of the creatment by the TDS reveals exactly that desired pattern 
of behavior mentioned' on the one hand the frequency of all activities on the 
level of knowledge decrease -~ even tiiose activities such as pupil response 
which involve a great amount of pupil participation. On the other hand, all 
activities on the anlytical level of thinking increase — even analytical 
lecturing which does not include any participation from the pupils' side. 

There is another way to combine feedback from a systematic observation system 
like TDS, while at the same time focusing on a specific skill as is done in 
mlcroteaching. It seems that this technique might give the best results. The 
Q "ure would be the following: each student would give a general lesson that 
be analysed with the aid of the TDS. The ratios computed from the TDS 
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would serve for diagnostic purpose, and make the student aware of the 
skills he has yet to master. After the diagnosis each student would receive 
treatment in the skill he needs most, and there would be focusing on that 
specific skill. Nevertheless, the next lesson for each student would 
also be analyzed by the TDS, and the supervision would again focus on the 
lesson as a whole and not on the specific skill practiced. This procedure 
should be repeated again and again, each time the TDS serves as a diagnostic 
system, and the skill practiced is the one in which the performance was the 
worst. The investigators intend to examine this combination during the 
next semester. 
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reacts reacts pupil pupil G 

lect. K iGct. A ^isks K asks A K resp.K res A 

for ^ 

, , , 0.24 6.39 0,08 0,24 0.33 1.40 2,10 3.63 j,;3 

ovariate 



I 



lect. - lectures; K - knowledge; A - analytical; Resp. - responds 
p O.OH F(l,123^^) - 6.85:- 



TABLE 2 

Means y F's and . .^ ^ Vs for tite nine posttest scores 



reacts reacts pupil pupil 





lect. K 


lect. A 


asks K 


iisks A 


K 


A 






res p. A 


G 




TDS 


TDS 


TDS 


TDS 


TDS 


TDS 


ins 




TDS 


TDS 




Yes No 


Yes No 


Yes No 


Yes No 


Yes No 


Yes No 


Yes N(i 




Yus No 


Yes No 


Yes 


25.7 A1.2 


4.9 1.7 


17.0 13.6 


11.3 3.2 


6.9 9.9 


5.1 0.7 


9.8 22. 


3 


9.] 2.6 


76 72 


:io 


1''.3 47. 4 


12.2 0.6 


7.4 13.2 


12.3 2.0 


4.5 8.1 


6.9 0.3 


10.5 18. 


6 


18.6 0.6 


90 60 


F 


48.27* 


34, 92 * 


1.00 


68.5fj" 


18.46" 


82. 33 


33.02 ' 




70 . 68* 


28 . 3* 


.2 


0.26 


0.19 




0. 23 


0. 11 


0. 38 


0.20 




0. 32 


0. 07 


F 


.032 


5.93 


16.41* 


0.00 


7.,70'' 


1. 33 


0.7 




6. 38 


0, 1 


.2 




0.03 


0. 10 




0.04 








0. 03 




F 


4.49 


11.12 


13.85* 


f). 93 


0. 22 


3.71 


]._._4 




_ _15 . G_8 '' __ 


J_7.50*__ 


o2 


0.02 


0.06 


0. 08 






O.OI 






0.07 


0. 04 



i^p^^ 0.01 ( F(l,123)— 6.35 } 
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